PACE-LM: Prompting and Augmentation for Calibrated Confidence Estimation with GPT-4 in Cloud Incident Root Cause Analysis

Major cloud providers have employed advanced AI-based solutions like large language models to aid humans in identifying the root causes of cloud incidents. Despite the growing prevalence of AI-driven assistants in the root cause analysis process, their effectiveness in assisting on-call engineers is constrained by low accuracy due to the intrinsic difficulty of the task, a propensity for LLM-based approaches to hallucinate, and difficulties in distinguishing these well-disguised hallucinations. To address this challenge, we propose to perform confidence estimation for the predictions to help on-call engineers make decisions on whether to adopt the model prediction. Considering the black-box nature of many LLM-based root cause predictors, fine-tuning or temperature-scaling-based approaches are inapplicable. We therefore design an innovative confidence estimation framework based on prompting retrieval-augmented large language models (LLMs) that demand a minimal amount of information from the root cause predictor. This approach consists of two scoring phases: the LLM-based confidence estimator first evaluates its confidence in making judgments in the face of the current incident that reflects its ``grounded-ness" level in reference data, then rates the root cause prediction based on historical references. An optimization step combines these two scores for a final confidence assignment. We show that our method is able to produce calibrated confidence estimates for predicted root causes, validate the usefulness of retrieved historical data and the prompting strategy as well as the generalizability across different root cause prediction models. Our study takes an important move towards reliably and effectively embedding LLMs into cloud incident management systems

STAIBT: Blockchain and CP-ABE Empowered Secure and Trusted Agricultural IoT Blockchain Terminal

The integration of agricultural Internet of Things (IoT) and blockchain has become the key technology of precision agriculture. How to protect data privacy and security from data source is one of the difficult issues in agricultural IoT research. This work integrates cryptography, blockchain and Interplanetary File System (IPFS) technologies, and proposes a general IoT blockchain terminal system architecture, which strongly supports the integration of the IoT and blockchain technology. This research innovatively designed a fine-grained and flexible terminal data access control scheme based on the ciphertext-policy attribute-based encryption (CP-ABE) algorithm. Based on CP-ABE and DES algorithms, a hybrid data encryption scheme is designed to realize 1-to-N encrypted data sharing. A "horizontal + vertical" IoT data segmentation scheme under blockchain technology is proposed to realize the classified release of different types of data on the blockchain. The experimental results show that the design scheme can ensure data access control security, privacy data confidentiality, and data high-availability security. This solution significantly reduces the complexity of key management, can realize efficient sharing of encrypted data, flexibly set access control strategies, and has the ability to store large data files in the agricultural IoT